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> SUBPIXEL RESOLUTION FROM MULTIPLE IMAGES 

Peter Cheeseman, RIACS, Bob Kanefsky, Recom, John Stutz, NASA, and Richard Kraft, Recom 

Multiple images taken from similar locations and under similar lighting conditions contain 
similar — but not identical — information. Slight differences in instrument orientation and position 
produces mismatches between the projected pixel grids. These mismatches ensure that any point on 
the ground is sampled differently in each image. If all the images can be registered with respect to 
each other to a small fraction of a pixel accuracy, then the information from the multiple images can 
be combined to increase linear resolution by roughly the square root of the number of images. In 
addition, the gray-scale resolution of the composite image is also improved. We describe methods for 
multiple image registration and combination, and discuss some of the problems encountered in 
developing and extending them. We display test results with 8:1 resolution enhancement, and Viking 
Orbiter imagery with 2:1 to 4:1 enhancements. 

Our problem is the inverse of computer graphics rendering. There, one is given a model of the 
relevant surfaces (shape and reflectivity), with associated viewing and lighting conditions. From this 
information, a graphics program projects an image of the scene. We have the inverse problem: given 
multiple images obtained under known lighting and viewing conditions, determine the most probable 
surface model subject to prior model constraints. Ideally, a planetary surface model would combine 
topographic information with multi-spectral reflectances at every point on the surface. Our current 
simplified approach is restricted to a single spectral band. We suppress topographic effects by requir- 
ing all images to be made from roughly the same direction under the same lighting conditions. Thus 
we combine the effects of varying ground slope and surface albedo into a 2-D model element 
(“mixel”) grid. This mixel grid is made at some fraction of the projected pixel resolution, e.g., 20m 
instead of 80m. 

Despite the above restrictions, we can achieve improved resolution in practical cases. For Viking 
Orbiter (VO), we can combine the overlapping margins and comers of high-resolution mosaics for a 
modest improvement. We can obtain dramatic improvement when dozens of images are available. 
Unfortunately, due to orbital mechanics, such sequences are always low-resolution. 

Lacking geographic truth, our current work employs a relative registration. The mixel grid is 
initialized by interpolating values from one of the pixel images. Thus this image provides the 
geographic control. Successive pixel images are then registered by minimizing a mismatch function 
over the parameter space of the appropriate mapping transformation. For each pixel the mismatch 
function calculates the sum of mixel contributions over the projected pixel area, weighted by the 
instrument point spread function (PSF). The mismatch is then the sum of squared errors between the 
predicted and observed pixel values. 

The choice of an appropriate geometric transform function is important. It must account for any 
inter-image distortions without introducing excessive degrees of freedom. For scanned images, rota- 
tion and translation are sufficient. With CCD detectors the addition of scale and shear will allow rela- 
tive mapping, while projective transform is needed for geographic correction. The vidicon tubes on 
the Viking Orbiters and Voyagers introduce local and image wide readout distortions that we model 
with second order coefficients. 

The final combined image is currently computed by setting each mixel to a weighted average of 
the pixel values influenced by that mixel via the PSF and registrations. This works surprisingly well, 
sufficiently to justify presenting the present technique. We are developing a full Bayesian solution to 
this inversion problem, to include PSF deconvolution and neighbor correlations. 

There are problems with the current system. The registration search can only find local minima 
in the mismatch error space, so it is necessary to manually pre-register images to within a few pixels. 
Practically, registration requires that there be several prominent features distributed over the image. 
Also, the mismatch function is very sensitive to edge interaction of the mixel and pixel grids. These 
can be avoided. 

In addition to readout distortion, the VO and Voyager vidicon images are subject to camera 
shading and uncertainties in exposure duration. Many pixels are blocked by data dropout and reseau 
marks. Others are corrupted by multi-bit transmission errors, quantization noise, blemishes, etc. 
These are all part of the camera model and could be reconstructed along with the surface model, but 
currently their effects are removed by preprocessing. 

We hope, in the future, to have a full geometric model that will allow us to take advantage of the 
multiple high-resolution coverage that can be found among the 50,000 VO images, as well as long- 
range Voyager flyby sequences of certain moons, such as Iapetus. 
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